Extracting associations and meanings of objects depicted in artworks through bi-modal deep networks

We present a novel bi-modal system based on deep networks to address the problem of learning associations and simple meanings objects depicted in "authored" images, such as ne art paintings drawings. Our overall processes both images associated texts order learn between individual objects, their identities abstract they signify. Unlike past net that describe infer predicates, our identies meaning-bearing ("signifiers") ("signifieds") well basic for target artworks. had precision 48% recall 78% with an F1 metric 0.6 curated set Dutch vanitas paintings, genre celebrated its concentration conveying meaning great import at time execution. developed tested but general methods can be applied other authored images.